Analyzing Empirical Quality Metrics of Deep Learning Models for Antimicrobial Resistance

Antimicrobial Resistance (AMR) is a growing concern in the medical field. Over-prescription of antibiotics as well as bacterial mutations have caused some once lifesaving drugs to become ineffective against bacteria. However, the problem of AMR might be addressed using Machine Learning (ML) thanks to increased availability of genomic data and large computing resources. The Pathosystems Resource Integration Center (PATRIC) has genomic data of various bacterial genera with sample isolates that are either resistant or susceptible to certain antibiotics. Past research has used this database to use ML algorithms to model AMR with successful results, including accuracies over 80%. To better aid future biologists and healthcare workers who may need a predictive model without the benefit of thousands of bacteria samples, this paper explores quantifying the empirical quality of some machine learning models—that is, quantifying how well a model will perform without prior knowledge of how the model performed on a training dataset.